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Let {Mn}n^o be a nonnegative Markov process with stationary 
' transition probabilities. The quasistationary distributions referred to 

' in this note are of the form 

ri ; QAix) = lim P(Af„ x\Mo ^A,Mi^A,...,M^^ A). 

Suppose that AIq has distribution Q,4 and define 
T^^ = min{n!A/„ > A,n^ 1}, 



the first time when M„ exceeds A. We provide sufficient conditions 
^ ' for ET^"* to be an increasing function of A. 

in 

Q!^ I 1. Introduction. Quasistationary distributions come up naturally in 

the context of first-exit times of Markov processes. Of special interest — in 
particular in statistical applications — is the case of a nonnegative Markov 
^3 ! chain, where the first time that the process exceeds a fixed level signals 

that some action is to be taken. The quasistationary distribution is the 
distribution of the state of the process if a long time has passed and yet no 
^ , crossover has occurred. 

^ ' Various topics pertaining to quasistationary distributions are existence, 



o 

O 



calculation, simulation, etc. For an extensive bibliography see Pollett (2008). 
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The topic addressed in this note deals with a certain aspect of the quasis- 
tationary distribution Qa as a function of A. Pollak and Siegmund (1986) 
have shown, under certain conditions, that if a stationary distribution Q 
exists, then — >■ Q as A — )■ oo. Here we study the behavior of the expected 
time of the first exceedance of ^ by a Markov process started at Qa, as a 
function of A. Specifically, we provide conditions under which it is increas- 
ing. Our interest stems from a result in changepoint detection theory, where 
a certain Markov chain that calls for a declaration that a change has taken 
place when a level A has been exceeded has certain asymptotic optimality 
properties if started at the quasistationary distribution (cf. Pollak, 1985; 
Tartakovsky et al., 2010). 

2. Results and Examples. Let (^2,^, P) be a probability space, and 
let {M^jJ^g be an irreducible Markov process defined on this space taking 
values in ^ C [0, oo) and having stationary transition probabilities p{t, x) = 

P{Mn+l ^ x\Mn = t). 

Let Ta = min{n|M„ > A;n^ 0}, and assume that: 
(CI) The quasistationary distribution 

Qa{x) = lim P(M„ ^ x\Ta > n) 

n— >oo 

exists for all A > Aq (for some < oo) and satisfies Qyi(O) = 0. 
(C2) p{s,x) is nonincreasing in s for all fixed x € 
(C3) p{ts, tx) is nondecreasing in t for all fixed s, x G 
(C4) p{s,x) / p{s, A) is nonincreasing in s for all fixed x € ^ A. 

(C5) p{ts,tx)/ p{ts,tA) is nondecreasing in t for all fixed s,x, € ^,x ^ A. 

Now regard the case where Mq has distribution and define 
T^-^ = min{n|M,, > A;n^l; Mq ~ Qa}- 

Theorem. Let the conditions (C1)~(C5) be satisfied. Then 

(i) QyA{yx) ^ QAix) for all y ^ I and all fixed x G x ^ A; 

(ii) ET^^ ^ ET^J^ for ally^l. 

Before proving the theorem, we provide examples that show that although 
the conditions (C1)-(C5) are restrictive, nevertheless they are satisfied in a 
number of interesting cases. 

Suppose {Mn}n^o obeys a recursion of the form 

M„+i = (^(M„) • A„+i, n = 0,l,..., 

where 



EXPECTATION OF THE FIRST EXIT TIME 



3 



(Dl) {Ajjjj:! are iid positive and continuous random variables; 
(D2) the distribution function F of Aj satisfies 




increases in t for fixed x € x ^ A; 



(D3) ip{t) is continuous, positive and nondecreasing in t; 

(D4) t/ip{t) is nondecreasing in t; 

(D5) if and F are such that P( Hm M„ = 0) = 0. 



In this example, 



Under these conditions. Theorem III. 10.1 of Harris (1963) can be applied to 
obtain existence of a quasistationary distribution. The conditions (D1)-(D5) 
are easily seen to imply the conditions (C1)-(C5). 

Condition (D2) is satisfied, for example, if the distribution function of 
log(Ai) is concave. 

Many "popular" Markov processes fit this model, some of which we now 
outline. 

(I) The exponentially weighted moving average (EWMA) processes: 



where ^ a < 1 and {^j} are iid random variables. Define M„ = e 
An = e?'\ Here ip{t) = 

(II) Let a > and (p{t) = t + a, so that M„+i = (M„ + a)A„+i. When a = 1 
and A„+i is a likelihood ratio (A„+i = fi{Xn+i) / fo{Xn+i) where Xi are iid), 
{Mn}n^o is a sequence of Shiryaev-Roberts statistics for detecting a change 
in distribution of Xi, from density /o to /i. The standard Shiryaev-Roberts 
procedure calls for setting Mq = 0, specifying a threshold A and declaring 
at Ta = min{n|M„ > A} that a change took place. A procedure T^'* that 
starts at a random point Mq ~ has asymptotic optimality properties (cf. 
Moustakides et al., 2010; Pollak, 1985; Tartakovsky et al., 2010). Another 
setting is where rj is the return on (one unit of) investment in the ith. 
period and Aj = 1 + r^, so that an investment of m units at the beginning 
of the ith period will be worth mAj at its end. If one invests a units at 
the beginning of the first period, reinvests the aAj units and adds another 
a units at the beginning of the second period, and continues this way (i.e., 
always reinvesting and adding a units at every period), then the process 



Yn+l = aYn + Cn+1, n ^ 0, 
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Mn+i = ip{Mn)An+i with (^(t) = t + a describes the scheme. 
(Ill) The random walk reflected from the zero barrier: 

^0 = 0, y„+i = + n = o,i,..., 

where {Zi} are iid, P{Zi < 0) > 0. Note that on the positive half plane the 
trajectory of the reflected random walk {Yn}n^o is identical to the trajectory 
of the Markov process {yn}n^o given by the recursion 

Yo* = o, y„Vi = (y;)+ + z„+i, n = o,i,... 

Therefore, if log A > one may operate with Y* instead of Yn and all 
conclusions will be the same. Define M„ = e^" and Aj = so that 

Mn+i = max(M„, l)A„+i, n ^ 0. 

Here (p{t) = max(l, t). This process describes a broad class of single-channel 
queuing systems (see, e.g., Borovkov, 1976). This setting can also be applied 
to the Cusum scheme for detecting a change in distribution, when Zi = 
log[/i(Xj)//o(Xj)] and Xi, fo and fi are as in (II). 

Proof of Theorem. Let {Unjn^o be a Markov process with stationary 
transition probabilities 

P([/„+i^x|C/„ = t) = 4^4, x^A, 

p[t,A) 

where A > is fixed and Uq has an arbitrary distribution (possibly degen- 
erate) on [0, A]. Let y > I and define Wn = yUn- 

Let {Kijn^o be a Markov process with Vq = Wq = yUo, having stationary 
transition probabilities 

P{Vn+i^x\Vn = t) = -^^ x^yA. 

p{t,yA) 

Clearly, the stationary distribution of {Vn} is QyA{x) and that of is 
QA{x/y). 
Since 



P{Vi ^ x\Vo) = ' > ^ 



p{Vo,yA) pflVo,A 



P (Ui ^ ix|[/o = IVo) = P{Wi ^ x\Wo = Vo 
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St 



it follows that Vi -< Wi (stochastically smaller). Therefore, one can construct 
a sample space on which Uq, Ui, Vq, Vi, Wq, Wi are all defined and such that 
Vi ^ Wi a.s. Write Vi = s,Wi = t where s ^ t ^ yA, s,t^Ji. Now 



so that V2 -< W2, and one can construct a sample space on which Uq, Ui, 



U2, Vo, Vi, V2, Wo, Wi, W2 are all defined and Vq = Wq, Vi ^ Wi, V2 ^ W2 



Continuing this inductively, one obtains a sample space on which {t/„,}, 
{^n}) {M^n} are all defined and Vn ^ Wn a.s. for all n ^ 0. Consequently, 
lim P{Vn > x) ^ lim P(W„ > x), i.e., QyA{yx) ^ Qa{x), accounting 

for (i). 

To prove (ii), note that both first exit times T^'* and T^J'^ are geomet- 
rically distributed random variables, so that 




St 



a.s. 



A 



1 



l-/(fp(s,A)dQ^(s) 



and 




1 



l-f^''p{s,yA)dQyA{s) 



Hence, it suffices to show that 
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Note that p{ds, t) ^ 0. Therefore, integrating by parts yields 

rvA yA ryA 

I p{s,yA)dQyA{s) = p{s,yA)QyA{s) - QyA{s)p{ds,yA) 
Jo ^0 

f-yA 

= p{yA, yA) - / QyA{s)p{ds, yA) (since QyA(O) = by (CI)) 
Jo 

ryA 

^p(:yA, yA)- QA{s/y)pids,yA) (by (i)) 



Jo 

p{yt,yA)QA{t) - / QiA{t)p{d{yt),yA) 
Jo 

A 

piyt,yA)dQA{t) 

r-A 







^ / p{t,A)dQAit) (by condition (C3)), 
Jo 

which completes the proof. □ 
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